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Some general considerations on the notion of entropy in physics are 
presented. An attempt is made to clarify the question of the differ- 
entiation between physical entropy (the Clausius-Boltzmann one) and 
quantities called entropies associated to Information Theory, which are 
in fact generating functionals for the derivation of probability distribu- 
tions and not thermodynamic functions of state. The role of them in 
the construction of the so-called Unconventional Statistical Mechanics, 
and the meaning and use of the latter, is discussed. Particular atten- 
tion is given to the situation involving far-from-equilibrium systems. 



1. INTRODUCTION: On Physical Entropy vs. 
Informational Entropies 

Entropy - The Curse of Statistical Mechanics?, wrote Herman Haken at some 
point in his book on Synergetics p]. G. N. Alekseev commented [2j that a 
popular-science author of the early twentieth century wrote that "Entropy is the 
name of the Queen's shadow. Face to face with this phenomenon, Man cannot 
help feeling some vague fear: Entropy, like an evil spirit, tries to diminish or 
annihilate the best creations of that gracious spirit. Energy. We are all under the 
protection of Energy and all potential victims of the latent poison of Entropy 
On the other hand, we can recall the sentence of the great Ludwig Boltzmann 
that "Thus, the general struggle for life is neither a fight for basic material ... nor 
for energy ... but for entropy becoming available by the transition from the hot 
sun to the cold earth" jS]. 

Moreover, Louis de Broglie |3] expressed that "En Thermodinamique clas- 
sique, on introduit pour enoncer le second principle de cette science la grandeur 
'Entropie' dont la signification physique restait si obscure que Henry Poincare la 
qualifiait de 'prodigeusement abstraite' ". As expressed by de Broglie entropy is 
related to the second law of Thermodynamics, and on that the French philosopher 
Henri Bergson j5j called this second law (or Entropy law) the most "metaphys- 
ical" of all the laws of Nature. Georgescu-Roegen [U] commented that from the 
epistemological point of view, the Entropy Law may be regarded as the greatest 
transformation ever suffered in physics, and that it is the basis of the economy 
of life at all levels. Also, that nobody is able to live without the sensation of the 
flux of entropy, that is, of that sensation that under diverse forms regulate the 
activities related to the maintenance of the organism. In the case of mammals 
this includes not only the sensations of cold and warm, but, also, the pangs of 
hunger or the satisfaction after a meal, the sensations of being tired or rested, and 
many others of the same type. 

Returning to de Broglie j3], he wrote that "C'est Boltzmann qui, en developant 
les idees de la Thermodinamique statistique, nos a donne le veritable sens de 
cette grandeur en montrant que I'entropie S de I'etat d'un corps est reliee a la 
probabilite P de cet etat par la celebre formule: S = ks InP" , and that "en 
Mecanique analytique [...] la veritable signification do principle de Hamilton est la 
suivante: Le mouvement d'un corps est celui que possede la plus grande probabilite 
thermodynamique dans les conditions auxquelles is est soumis. Je pense que cette 
conception de la nature profunde du principle de Hamilton jette une fiot de lumiere 



sur sons veritable sens, analogue a celui que jette la formule de Boltzmann sur la 
signification de I'entropie" . 

The law of increase of entropy addresses the question of irreversibility of pro- 
cesses in nature, that is, the famous "time-arrow" according to which all macro- 
scopic events have a preferred time direction. The question has recently been ad- 
dressed in two relevant papers in Physics Today. On the one hand, Joel Lebowitz 
[7] expressed that "Boltzmann's thoughts on this question have withstood the 
test of time. Given the success of Ludwig Boltzmann's statistical approach in 
explaining the observed irreversible behavior of macroscopic systems in a manner 
consistent with their reversible microscopic dynamics, it is quite surprising that 
there is still so much confusion about the problem of irreversibility" . In the other 
article, by Elliot Lieb and Jacob Yngvason jH], it is raised the point that "the ex- 
istence of entropy, and its increase, can be understood without reference to either 
statistical mechanics or heat engines". 

Points of view associated to the latter one go back in time, and it can be 
mentioned the approach of Elias Gyftopoulos et al. jHllini: The basic idea seems 
to consist in that, instead of regarding mechanics and thermodynamics as two 
different theories, belonging to two different levels of fundamentality, can be con- 
sidered a new hypothesis, namely, that mechanics and thermodynamics are two 
particular aspects of a more general fundamental theory ^T]. On entropy it has 
been noticed jH] that "In his extensive and authoritative review, Wehrl writes 
'It is paradoxical that although entropy is one of the most important quantities 
in physics, its main properties are rarely listed in the usual textbooks on statisti- 
cal mechanics'... The lack of specificity has resulted in a plethora of expressions 
purporting to represent the entropy of thermodynamics (emphasis is ours), and 
perhaps influenced von Newmann to respond to Shannon's question 'what should 
I call Pi In Pi?' by saying 'You should call it 'entropy' for two reasons: first, 
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the function is already in use in thermodynamics under that name; second, and 
more importantly, most people don't know what entropy really is, and if you 
use that word entropy you will win every time' ". Moreover, ^3] that quan- 
tum theory admits not only probability distributions described by wave functions 
or projectors, but also probability distributions that are described by density 
operators g which are not statistical averages of projectors. Said differently, g 
can be represented only by a homogeneous ensemble, namely, an ensemble every 
member of which is characterized by the same g as the whole ensemble. For pro- 
jectors (wave functions) the concept of a homogeneous ensemble was conceived 
by von Neumann. This discovery plus all the requirements that the entropy of 



thermodynamics must satisfy both for reversible and irreversible processes yield 
that the only expression for entropy is 5* = —ksTr {gin g}, provided that g is 
representable by a homogeneous ensemble. In addition, the discovery requires a 
new complete equation of motion different from the Schrodinger equation. Such 
an equation has been conceived by Beretta ^1]. In both, an unified quantum 
theory of mechanics and thermodynamics, and in a novel exposition of thermody- 
namics without reference to quantum mechanics, it is proved that entropy is an 
intrinsic (inherent) property of each constituent of any system (both macroscopic 
and microscopic), in any state (both thermodynamic equilibrium and not ther- 
modynamic equilibrium) in the sense that inertial mass is an intrinsic property 
of each constituent of any system in any state ^Sl- We also noticed that, Edwin 
Jaynes showed that the expression above can be identified with Clausius' entropy 
jl6j . and with Boltzmann-Planck expression for isolated systems in equilibrium, 
namely, the celebrated S = ks InW . 

On this, in a recent journalistic article by A. Cho in Science ^7] we can read 
that "Near the middle of Vienna's sprawling Central Cemetery stands the im- 
posing tomb of Ludwig Boltzmann, the 19th century Austrian physicist who first 
connected the motions of atoms and molecules to temperature, pressure, and other 
properties of macroscopic objects. Carved in the gravestone, a single short equa- 
tion serves as the great man's epitaph: S = InW . No less important than Ein- 
stein's E = mc'^, the equation provides the mathematical definition of entropy, a 
measure of disorder that every physical system strives to maximize. The equation 
serves as the cornerstone of "statistical mechanics", and it has helped scientists 
decipher phenomena ranging from the various states of matter to the behavior of 
black holes to the chemistry of life" . In continuation A. Cho comments on the ex- 
istence of proposals of new and supposedly extended forms of Boltzmann entropy, 
apparently necessary to deal with nonextensive systems. This does not seem to 
be correct jTHl 112] , and we are facing here the question posted before that the lack 
of specificity has resulted in a plethora of expressions purporting to represent the 
entropy of thermodynamics jU]. The propagation of this idea about a more gen- 
eral and universal expression for entropy, is a result of the confusion arising from 
the fact that the proposed ones are informational entropies (that is, generating 
functionals of probability distributions): They provide a practical and useful tool 
for handling problems where the researcher does not have access to the complete 
necessary information on the relevant - for the problem in hands - characteristics 
of the system, for properly applying the well established Boltzmann-Gibbs quite 
general, and physically and logically sound, theory. That is, it provides a way 



out to contour the failure we do have to satisfy Fisher's Criterion of Sufficiency 
in statistics of 1922 [20], as shown as we proceed. There are infinitely-many pos- 
sible generating functionals that can be introduced for that purpose, what was 
done around the middle of past century by a school of statisticians. This lead 
to the development of unconventional statistics, which are useful for making pre- 
dictions in the cases when the general and extremely successful Boltzmann-Gibbs 
statistics has its application impaired once, as noticed, the researcher cannot sat- 
isfy Fisher's Criterion of Sufficiency [20]; that is, we are unable to introduce in 
the calculations a proper description of the characteristics of the system that are 
relevant for the experiment under consideration. 

As said a large number of possibilities (in principle infinitely-many) can be 
explored, and Peter Landsberg quite properly titled an article Entropies Calore! 
|21j . An infinite family is the one that can be derived from Csiszer's general 
measure of cross-entropies (see for example |221); other family has been proposed 
by Landsberg [23]; and specific informational entropies are, among others, the 
one of Skilling [21] - which can be used in mathematical economy -, and of 
Kaniadakis [2S] who used it in the context of special relativity [26]. They, being 
generating functionals of probability distributions, give rise to particular forms 
of statistics as the one of next section which we have dubbed Unconventional 
Statistical Mechanics [23 I2H] ; we do also have so-called Superstatistics proposed 
by C. Beck and E. G. D. Cohen for dealing with nonequilibrium systems with a 
stationary state and intensive parameter fluctuations [23 130] ; what can be called 
Kappa Statistics [2312115 and so on. 

What we do actually have behind this question is the possibility to introduce 
of such sophisticated method in physics, more precisely in statistical mechanics, 
what we do try to describe in continuation. 

2. UNCONVENTIONAL STATISTICAL MECHANICS 

We begin calling the attention to the fact that Statistical Mechanics of many- 
body systems has a long and successful history. The introduction of the concept 
of probability in physics originated mainly from the fundamental essay of Laplace 
of 1825 [3T], who incorporated and extended some earlier seminal ideas (see for 
example [22] )• As well known. Statistical Mechanics attained the status of a well 
established discipline at the hands of Maxwell, Boltzmann, Gibbs, and others, and 
went through some steps related to changes, not in its fundamental structure, but 
just on the substrate provided by microscopic mechanics. Beginning with classical 



dynamics, statistical mechanics incorporated - as they went appearing in the 
realm of Physics - relativistic dynamics and quantum dynamics. Its application 
to the case of systems in equilibrium proceeded rapidly and with exceptional 
success: equilibrium statistical mechanics gave - starting from the microscopic 
level - foundations to Thermostatics, and the possibility to build a Response 
Function Theory. Applications to nonequilibrium systems began, mainly, with 
the case of local equilibrium in the linear regime following the pioneering work of 
Lars Onsager (see also [Ml)- 

For systems arbitrarily deviated from equilibrium and governed by nonlinear 
kinetic laws, the derivation of an ensemble-like formalism proceeded at a slower 
pace than in the case of equilibrium, and somewhat cautiously, with a long list 
of distinguished scientists contributing to such development. It can be noticed 
that Statistical Mechanics gained in the fifties an alternative approach sustained 
on the basis of Information Theory jn2lESlESlEZlEHlEniliniiIlll2]: It invoked 
the ideas of Information Theory accompanied with ideas of scientific inference 
I44j ■ and a variational principle (the latter being Jaynes' principle of maxi- 
mization of informational uncertainty - also referred-to as informational-entropy 
" and called MaxEnt for short), compounding from such point of view a the- 
ory dubbed as Predictive Statistical Mechanics jSHESHHEllSZlEHlEnilinillS]. 
It should be noticed that this is not a new paradigm in Statistical Physics, but 
a quite useful and practical variational method which codifies the derivation of 
probability distributions, which can be obtained by either heuristic approaches 
or projection operator techniques [IHl EZl EHl EH] ■ It is particularly advantageous 
to build nonequilibrium statistical ensembles, as done here, when it systematizes 
the relevant work on the subject that renowned scientists provided along the past 
century. The informational-based approach is quite successful in equilibrium and 
near equilibrium conditions jSSl EEl HH 112]; and in the last decades has been, 
and is being, also applied to the construction of a generalized ensemble theory for 
systems arbitrarily away from equilibrium WJ\ EHl IH] • The nonequilibrium 
statistical ensemble formalism (NESEF for short) provides mechanical-statistical 
foundations to irreversible thermodynamics (in the form of Informational Sta- 
tistical Thermodynamics - 1ST for short (SUl El E21 EH] ) , a nonlinear quantum 
kinetic theory ^\ HSl |33j and a response function theory jlSl of a large scope 
for dealing with many-body systems arbitrarily away from equilibrium. NESEF 
has been applied with success to the study of a number of nonequilibrium sit- 
uations in the physics of semiconductors (see for example the review article of 
Ref. [Sni) and polymers j^Zj, as well as to studies of complex behavior of boson 



systems in, for example, biopolymers (e.g. Ref. [3H]). It can also be noticed 
that the NESEF-based nonlinear quantum kinetic theory provides, as particular 
limiting cases, far-reaching generalizations of Boltzmann [^1] , Mori (together with 
statistical foundations for Mesoscopic Irreversible Thermodynamics 60]) W^, and 
Navier-Stokes |^ equations and a, say. Informational Higher-Order Hydrodynam- 
ics, linear j^S] and nonlinear jMj . 

NESEF is built within the scope of the variational method on the basis of the 
maximization of the informational-entropy in Boltzmann-Gibbs-Shannon-Jaynes 
sense, that is, the average of minus the logarithm of the time-dependent - i.e. 
depending on the irreversible evolution of the macroscopic state of the system 
- nonequilibrium statistical operator. It ought to be further emphasized that 
informational- entropy - a concept introduced by Shannon - is in fact the quan- 
tity of uncertainty of information, and has the role of a generating functional for 
the derivation of probability distributions (for tackling problems in Communica- 
tion Theory, Physics, Mathematical Economics, and so on). There is one and only 
one situation when Shannon- Jaynes informational-entropy coincides with the true 
physical entropy of Clausius in thermodynamics, namely, the case of strict equi- 
librium; e.g. 1^ ins ISni • For short, we shall refer to informational-entropy as 
infoentropy. As already noticed the variational approach produces the well estab- 
lished equilibrium statistical mechanics, and is providing a satisfactory formalism 
for describing nonequilibrium systems in a most general form. This Boltzmann- 
Gibbs Statistical Mechanics properly describes the macroscopic state of condensed 
matter systems, being a well established one and logically and physically sound, 
but in some kind of situations, for example, involving nanometric-scale systems 
with some type or other of fractal-like structures or systems with long-range space 
correlations, or particular long-time correlations, it becomes difficult to apply be- 
cause of a deficiency in the proper knowledge of the characterization of the states 
of the system in the problem one is considering (at either the microscopic or/and 
macroscopic or mesoscopic level). This is, say, a practical difficulty (a limitation 
of the researcher) in an otherwise general and highly successful physical theory. 

In fact, in a classical and fundamental paper of 1922 [201 by Sir Ronald Fisher, 
titled "On the Mathematical Foundations of Theoretical Statistics" , are presented 
the basic criteria that a statistics should satisfy in order to provide valuable re- 
sults, that is, reliable predictions. In what regards present day Statistical Me- 
chanics in Physics two of them are of major relevance, namely the Criterion of 
Efficiency and the Criterion of Sufficiency. This is so because of particular con- 
straints that impose recent developments in physical situations involving small 



systems (nanotechnology, nanobiophysics, quantum dots and heterostructures in 
semiconductor devices, one-molecule transistors, fractal electrodes in microbatter- 
ies, and so on), where on the one hand the number of degrees of freedom entering 
in the statistics may be small, and on the other hand boundary conditions of a 
fractal-like character are present which strongly influence the properties of the 
system, what makes difficult to introduce sufficient information for deriving a 
proper Boltzmann-Gibbs probability distribution. Other cases when sufficiency 
is difficult to satisfy is the case of large systems of fluids whose hydrodynamic 
motion is beyond the domain of validity of the classical standard approach. It 
is then required the use of a nonlinear higher-order hydrodynamics, eventually 
including correlations and other variances (a typical example is the case of turbu- 
lent motion). Also we can mention other cases where long-range correlations have 
a relevant role (e.g. velocity distribution in clusters of galaxies at a cosmological 
size, or at a microscopic size the already mentioned case of one-molecule transis- 
tors where Coulomb interaction between carriers is not screened and then of long 
range creating strong correlations in space). 

Hence, we may say that the proper use of Boltzmann-Gibbs statistics is simply 
impaired because of either a great difficulty to handle the required information 
relevant to the problem in hands, or incapacity on the part of the researcher to have 
a correct access to such information. Consequently, out of practical convenience 
or the force of circumstances, respectively, a way to circumvent this inconveniency 
in such kind of "anomalous" situations, consists to resort to the introduction of 
modified forms of the informational- entropy, that is, other than the quite general 
one of Shannon- Jaynes which leads to the well established statistics of Boltzmann- 
Gibbs. These modified infoentropies which are built in terms of the deficient 
characterization one does have of the system, are dependent on parameters - 
called information-cntropic indexes, or infoentropic indexes for short with the 
understanding that refer to the infoentropy. We reiterate the fundamental fact that 
these infoentropies are generating functional for the derivation of probabilities 
distributions, and are not at all to be confused with the physical entropy of the 
system. 

As already noticed, this alternative approach originated in the decades of the 
1950's and 1960's at the hands of theoretical statisticians, being extensively used 
in different disciplines (economy, queueing theory, regional and urban planning, 
nonlinear spectral analysis, and so on). Some approaches were adapted for use 
in physics, and we present here an overall picture leading to what can be called 
Unconventional Statistical Mechanics (USM for short and fully described in Refs. 



123 IIHI), consisting, as said before, in a way to patch the lack of knowledge of 
characteristics of the physical system which are relevant for properly determining 
one or other property (see also P. T. Landsberg in Refs. [21] and [2H]) thus 
impairing the correct use of the conventional one. 

Use of the variational MaxEnt for building NESEF provides a powerful, prac- 
tical, and soundly-based procedure of a quite broad scope, which is encompassed 
in what is sometimes referred-to as Informational- Entropy Optimization Princi- 
ples (see for example Ref. 22] )• To be more precise we should say constrained 
optimization, that is, restricted by the constraints consisting in the available in- 
formation. Such optimization is performed through calculus of variation with 
Lagrange's method for finding the constrained extremum being the preferred one. 

Jaynes' variational method of maximization of the informational-statistical 
entropy is connected - via information theory in Shannon-Brillouin style - to a 
principle of maximization of uncertainty of information. This is the consequence 
of resorting to a principle of scientific objectivity [1111121 , which can be stated as: 
Out of all probability distributions consistent with a given set of constraints, we 
must take the one that has maximum uncertainty. Its use leads to the construction 
of a Gibbs' ensemble formalism, recovering the traditional one in equilibrium 
|85t l86] HT] , and allowing for the extension to systems far from equilibrium ^2] 

uniiiziiiHiiini. 

Jaynes' MaxEnt is a major informational-entropy optimization principle re- 
quiring, as noticed, that we should use only the information which is accessible 
but scrupulously avoiding to use information not proven to be available. This is 
achieved by maximizing the uncertainty that remains after all the given informa- 
tion has been taken care of. 

Jaynes' MaxEnt aims at maximizing uncertainty when subjected to a set of 
constraints which depend on each particular situation (given values of observables 
and theoretical knowledge). But uncertainty can be a too deep and complex con- 
cept for admitting a unique measure under all conditions: We may face situations 
where uncertainty can be associated to different degrees of fuzziness in data and 
information. As already noticed, this is a consequence, in Statistical Mechanics, of 
a lack of a proper description of the physical situation. This corresponds to being 
violated the Criterion of Sufficiency in the characterization of the system ( "the 
statistics chosen should summarize the whole of the relevant information supplied 
by the sample") [20] ■ This could occur at the level of the microscopic dynamics 
(e.g. lack of knowledge of the proper eigenstates, all important in the calcula- 
tions), or at the level of macroscopic dynamics (e.g. when we are forced, because 



of deficiency of knowledge, to introduce a low-order truncation in the higher-order 
hydrodynamics that the situation may require). Hence, in these circumstances it 
may arise the necessity of introducing alternative types of statistics, with the 
accompanying indexed (or structural) informational- entropies, {infoentropies for 
short ) different of the quite general and well estabhshed one of Boltzmann-Gibbs, 
as it has been discussed in the first section. 



TABLE I: Informational-Statistical Entropies 



Conventional (Universal) ISE 

Boltzmann-Gibbs-Shannon-Jaynes ISE | —Tr{g\ng} 
(from Kiilback-Leibler measure) 

Unconventional (entropic-index-dependent) ISEs 

-^^Tr{g--g} 



From Havrda-Charvat measure 



a > and a ^ 1 



From Sharma-Mittal measure < "^'^ 

a > 1, l3 < 1 or a < 1, f3 > I 



From Renyi measure 



From Kapur measure 



In Tr{f?n 
a > and a 7^ 1 

-^[inTrig^j-lnTrig"}] 
a > 0, P > and a ^ [3 



Applying MaxEnt to any of these unconventional infoentropies we obtain 
a probability distribution deemed appropriate for the given problem in hands, 
namely, either the conventional probability distribution when Shannon- Jaynes in- 
foentropy is used, or the so-called in Pearsons' nomenclature heterotypical proba- 
bility distributions when other infoentropies are used. There are infinitely-many of 
these infoentropies, which, as said, are dependent on one or more parameters (the 
infoentropic indexes), and in Table I we list four of these informational-statistical 
entropies (ISE), plus the Shannon- Jaynes one (of Boltzmann-Gibbs type) which 
is parameter free as it should for being the most general one fl^ l?fl US] . In Table 
I is the constant value of probability in the uniform distribution, and a and 
P infoentropic indexes (open parameters). 

Renyi's approach appears to be a particularly convenient one for dealing with 
fractal systems as discussed in Ref. [65j, where it is pointed out that predictions 
obtained resorting to the approach of maximization in Shannon- Jaynes approach 
including fractality can be equivalently obtained using Renyi's approach ignor- 
ing fractality. Renyi's ISE has been studied by Takens and Verbitski jHTj, and a 
variation of it is Hentschel-Procaccia infoentropy [UH]- For the Havrda-Charvat 



structural a-entropy, one akin to the case a = 2 has been considered by I. Pri- 
gogine in connection with practical and theoretical difficulties with Boltzmann 
ideas when extending them from the dilute gas to dense gases and liquids [69] . 
Prigogine argues that to cope with such situations one would need a statistical 
expression of entropy that depends explicitly on correlations, as is the case of 
the Havrda-Charvat structural a-entropy for a = 2 (also in the case of Renyi 
infoentropy). 

It can be noticed that taking f3 = 1 reduces Kapur ISE to the one of Renyi, and 
Sharma-Mittal ISE to the one of Havrda-Charvat. Moreover, taking also a = 1, is 
obtained an ISE which is of the form of Boltzmann-Gibbs-Shannon-Jaynes ISE. 
What we do have in these ISE's, or in any other one of the infinitely-many which 
are possible, is that when the adjustment of the parameters (the infoentropic 
indexes) on which they depend - let it be in a calculation or as a result of the 
comparison with the experimental data - produces Boltzmann- Gibbs result, this 
gives an indication that the principle of sufficiency is being satisfied, i.e., for such 
particular situation the description of the system we are doing includes all the 
relevant characterization that properly determines the physical property that is 
measured in the given experiment being analyzed. 

Moreover, we again stress the fundamental fact that the structural informatio- 
nal-entropies (quantity of uncertainty of information) are not to be confused with 
the Clausius-Boltzmann physical entropy: There is one and only one case when 
there is an equivalence, consisting of Shannon infoentropy when the system is 
strictly in equilibrium P^I^IH3] . Boltzmann-Gibbs-Shannon-Jaynes nonequilib- 
rium informational entropy and its role in NESEF is extensively discussed in Refs. 



It is quite relevant to notice that for each kind of statistical entropy it is 
necessary in an ad hoc manner, to introduce definitions of average values of ob- 
servables with particular forms, what is required to obtain a posteriori consistent 
results. For the case of Kullback-Leibler measure, or Shannon- Jaynes statisti- 
cal informational-entropy, we must use the usual expression, i.e. the average of 
quantity A is given by 



while for the case of Renyi ISE, needs be introduced an average of the form 



gHlESllZni- 




(1) 



(A) = Tr {AV, {g}] 



(2) 



that is, in terms of the so-called escort probability of order 7 [TH [72] 



V^{g} = gyTr{0''} , (3) 

which is also the one to be used in the case of Havrda-Charvat statistics. A 
detailed discussion on the role of escort probability is given in Ref. [2Z]- 

We call the attention to the fact that USM is to be based on the use of both defi- 
nitions, namely, the heterotypical probability distribution and the escort probability 
(notice that for probability distributions other than Renyi and Havrda-Charvat 
other definitions of escort probabilities should be introduced). The role of the 
escort probability accompanying the heterotypical-probability distribution is that 
both complement each other in order to redefine, in the sense of weighting, the 
values of the probabilities associated to the physical states of the system fTTj . 

Let us now consider the use of Renyi informational entropy, i.e. 

SAt) = ^ In Tr { [^,(t,0)n , (4) 

in the case of a fluid of single molecules driven out of equilibrium and in contact 
with ideal reservoirs; a more general situation is discussed in [27] . We recall that 
a recent application of Renyi's statistics for dealing with (multi)fractal systems 
is presented by Jizba and Arimitzu [65j: there it is addressed the question on 
how Renyi's approach appears as a quite convenient one in such cases. Further 
considerations on Renyi's approach can be consulted in the articles by Hentschel 
and Procaccia and Takens and Verbitski j|67j. 

We proceed to derive the nonequilibrium statistical operator in Renyi's ap- 
proach, beginning to look for the "instantaneously frozen" auxiliary statistical 
operator which, in the situation above described, consists of the product of the 
one of the system, say {t,0), times the constant one of the thermal bath and 
ideal reservoirs. Let us now look for g^ [t, 0) which follows by maximizing Sa of 
Eq. ([U) subjected to the conditions of normalization 

Tr{-g^it,0)} = l , (5) 

and the constraints consisting of the average values, say Qj (r, t), as defined by 
Eq. ([21) , of the basic dynamical variables, say |-Pj (r)|, namely 

g,(r,t) = rr(p,(r)5a{^(t,0)}\ , (6) 



where 

P.{^(t,0)} = [^Jt,0)r /Tr{[^Jt,0)n (7) 

is the corresponding escort probabihty [ZT1IZ2] (cf. discussion after Eq. (jH)) above); 
it can be observed that the order of the escort probabihty is the same infoentropic 
index in Renyi's informational entropy [7T] . 
It fohows that 



l + ia-l)J2 j d'r (r, t) AP, (r, t) 



where 



AP,(r,t)=P,(r)-g,(r,t) 
with Qj {r,t) given in Eq. ©, 



(8) 



(9) 



1 + (« - 1) J] y rfV Fj^ (r, t) APj (r, t) 



(10) 



ensures the normahzation condition, and Fj^ are the Lagrange muhiphers that the 
variational method introduces, which are related to the basic variables through 
Eq. ©. 

In terms of the auxiliary g^, the statistical distribution is given by [IHlllZlllHl 



'11] 



where, we recall. 



{f, t'-t)=exp\—- {f -t)H\-Q^ {t\ 0) exp \ - {t' - t) H 



ih 



ih 



'121 



and e is a positive infinitesimal that goes to zero after the trace operation in the 
calculation of averages has been performed. The statistical distribution of Eq. 
(fTT|l satisfies the Liouville equation 



d 1 r 



(13) 



with the presence of the infinitesimal source introducing Bogoliubov's symmetry 
breaking procedure (quasiaverages) [73], in the present case the one of time re- 
versal and in that way are discarded the advanced solutions of the full Liouville 
equation. Thus, the retarded solutions have been selected, and, a posteriori, this 
is transmitted to the kinetic equations producing a fading memory and irreversible 
behavior (cf. Refs. gHlElj)- 

We also call the attention to the fact that for average values, as given by Eq. 
(121) , we then have 

^A) = Tr{AV^,{g^At)}] , (14) 

where 

1^.e{Qae{t)} = Q^aeit) /Tr{g2At)} , (15) 

and it is implicit the limit e — after the calculation of traces has been performed. 
Because of the boundary condition g^^ (to) = g^ (to, 0) (tg —oo) [IHl EZl 

EHl HH], we have that Vae {gae (^o)} =Va {Qa (to, 0)}, where Va is given by Eq. 
(|7j). For e — > 0, f)^^ satisfies a true Liouville equation [cf. Eq. (fT5|) ]. and so 
does "Dae, and we recall that the infinitesimal source on the right-hand side of 
Eq. (fT^ is selecting the retarded solutions of the true Liouville equation (via, 
then, Bogoliubov's method of quasiaverages, as previously noticed). Hence, for 
the given initial condition and the imposition of discarding the advanced solutions, 
l^at {Qae (^)} ^^^o satisfics a modified Liouville equation, and we can write 

l^ae {Qae (t)} =Vc. {Qo. (t, 0)} + (t) , (16) 

where Va {Qait,0)} is given by Eq. (|7j), and 

t 

it) = - I it\t' - t)} . (17) 

— oo 

Introducing Eq. (fT^ into Eq. (HH), we can see that the averages are com- 
posed of an "instantaneously frozen" (at time t) contribution, plus a contribution 
associated to the irreversible processes and including historicity. For the basic 
dynamical quantities, and only for them, it follows that 

Q, (r,t) = Tr [Pp^e {Q^e H)}] = Tr |p, Va {Qa (^,0)}| . (18) 



with, as already noticed, being implicit the limit of e going to +0 to be taken after 
the calculation of the trace operation has been performed [IHl EH EHl EHl ■ 

After the nonequilibrium distribution using a heterotypical index-dependent 
information- 
al-entropy has been derived, next step - like done in the conventional case 
EHl HH HOI m] ^ should consists in deriving for arbitrarily far-from-equilibrium 
systems, a nonlinear quantum kinetic theory, a response function theory, and, of 
course, a systematic study of experimental results, that is, a full collection of mea- 
surements of diverse properties of the system, amenable to be studied in terms of 
structural (infoentropic-index dependent) informational-entropies, what is funda- 
mental for the validation of the theory (see for example Refs. [^173117^1771 175]). 

3. IRREVERSIBLE THERMODYNAMICS AND 
ENTROPY 

As know, nonequilibrium thermodynamics is involved with finite and irreversible 
processes, and, for that reason, it is also denominated Irreversible Thermody- 
namics. As a general rule special interest is currently focused on the case of 
open systems, which are coupled to external sources which supply them energy 
and matter. This manifests itself in macroscopic changes of the thermodynamic 
variables which naturally become dependent on space position and on time. 

The principal characteristic of the processes involved, whether they are station- 
ary or time dependent, is that they evolve with a positive production of internal 
entropy. Here we make contact with the second fundamental principle of ther- 
modynamics. According to Planck jT^I, "The second law of thermodynamics is 
essentially different from the first law, since it deals with a question in no way 
touched upon by the first law, viz., the direction in which a process takes place 
in nature". The second law has several equivalent formulations: the one due to 
Clausius refers to heat conduction, and, again according to Planck [79j, it can be 
expressed as "heat cannot by itself pass from a cold to a hot body. As Clausius 
repeatedly and expressly pointed out, this principle does not merely say that heat 
does not flow from a cold to a hot body - that is self-evident, and is a condition 
of the definition of temperature - but it expressly states that heat can in no way 
and by no process be transported from a colder to a warmer body without leaving 
further changes, i.e. without compensation". 

The concept of irreversibility clearly appears in Clausius' work, who can be 
considered the founder of Thermodynamics as an autonomous and unified sci- 



ence. His presentation of the second law appears precisely as a criterion for evo- 
lution, governing irreversible behavior. It implies the definition and use of the 
extremely difficult concept of the title, namely, entropy. Using the usual notation, 
for an infinitesimal process the state function entropy S suffers a modification 
dS = dQ/T if the process is reversible, while for irreversible processes there fol- 
lows dS > dQ/T, where dQ is the heat exchange in the process and T the absolute 
temperature, and D stands for a nonexact differential (thus, 1/T acts as an inte- 
grating factor). The last inequality can be expressed in the form of the balance 
equation 

(19) 

with <)Q' > 0. In this form of the second law, the quantity DQ', called by Clau- 
sius uncompensated heat, can be considered as a first evaluation of the degree of 
irreversibility of a natural process. 

Equilibrium thermodynamics describes states of matter that are greatly priv- 
ileged: Planck [TH] has emphasized that the second law distinguishes among the 
several types of states in nature, some of which act as attractors to others: Ir- 
reversibility is an expression of this attraction. For systems together with the 
reservoirs to which they are connected there is an attraction towards equilibrium 
(thermodynamic potentials at a minimum value compatible with the constraints 
imposed by the reservoirs). In nonequilibrium systems, while in a stationary state 
in a linear regime near equilibrium, the attractor is the state of minimum internal 
entropy production 

According to Eq. (fT^ . the change of entropy is composed of two separate 
types of contributions: the term DQ/T related to the exchange of heat with the 
surroundings, expressed as dgS, and the term DQ' /T, due exclusively to the irre- 
versible processes that develop in the interior of the system, expressed as diS so 
we can write 

dS = d^S + d,S, with d,S>0. (20) 

Equation (j2(Jj) provides the framework for an entropy equation of balance, equiva- 
lent to Eq. (fTU|) . Let us emphasize that the term dgS involves all the contributions 
resulting from exchanges (energy, matter, etc.) with the environment. 

After what has been manifested in Section 1, it is certainly a truism to say that 
entropy has a very special status in physics, expressing, in a very general way, the 
tendency of physical systems to evolve in an irreversible way, characterizing the 
eventual attainment of equilibrium, and given a kind of measurement of the order 
that prevails in the system. It is important to further stress that it is a very well 



established concept in equilibrium situations, however requiring an extension and 
clear comprehension in the case of open systems, mainly in far-from-equilibrium 
conditions, a question that remains controversial. On the basis of the use of 
entropy as a state function, the properties of systems in equilibrium are very well 
described when in conjunction with the two fundamental laws. Let us add some 
additional consideration to what have been said in Section 1. 

Lawrence Sklar (HUI has noted that the concept of entropy is the most purely 
thermodynamic concept of all, and J. Bricmont [SI] has commented that there 
is some kind of mystique about entropy. Again according to Sklar, given the 
abstractness of entropy and its high place up in the theory as well as its unrelat- 
edness to immediate sensory qualities or primitive measurements (as temperature 
is related to these), it is not surprising than in seeking the statistical mechanical 
correlate of nonequilibrium thermodynamic entropy we have the least guidance 
from the surrounding embedding theory. Exists an openness in what to choose 
as the surrogate for entropy. Thus, it should be expected to arise a wide variety 
(plethora) of "entropies" , each functioning well for the specific purposes for which 
it was introduced. J. Meixner jH21 asks: Is the concept of nonequilibrium entropy 
superfluous?, for in continuation to comment that one is so much accustomed to 
the concept of entropy that one would like to retain it as a quantity of physical 
significance. He also points to the difficulty of a definition if one does not have 
a clearly defined physical state of the system. This is the main difficulty, as also 
pointed out by Bricmont in that we may define as many entropies as we can 
find sets of macroscopic variables. Also, with the coarse-graining procedure there 
is not a sharp distinction between microscopy and macroscopy, passing through 
mesoscopy, so that we can arrive at many values of the say "entropy", including 
arriving at the zero value when a complete microscopic description is given, and 
we have a pure mechanical description and no thermodynamics exists f |82[ I83j. 
see also Jaynes in JH])- Jaynes rightly says that he does not know what is the 
entropy of a cat; the problem being that we are unable to precisely define a set of 
macrovariables that properly specify the thermodynamic state of the cat. 

At this point it is worth to present, with some modifications, several quite 
appropriate remarks made by Bricmont in the cited reference, which we roughly 
summarize here: 

(i) These entropies are not subjective but objective as are the corresponding 
macroscopic variables. Called "anthropomorphic" by Jaynes following Wigner, 
they may be referred to as "contextual", i.e. they depend on the physical situation 
and on its level of description. 



(ii) The "usual" or "traditional" entropy of Clausius corresponds to the par- 
ticular choice of macroscopic variables for a free monatomic gas in equilibrium 
(energy, specific volume, number of particles). The derivative with respect to the 
energy of that entropy defines the reciprocal of the Kelvin's absolute temperature. 

(iii) The second law in the form "Entropy increases" becomes undetermined: 
which entropy? Between two states of equilibrium is the Clausius' one. Otherwise 
is not clear. 

(iv) Whichever the chosen functional form for an entropy, in most cases is hard 
to compute or estimate. One needs to begin with the equations of evolution for 
the chosen basic variables and solve them for appropriate initial and boundary 
conditions. Moreover, irreversibility as characterized by some 7i-theorem - as the 
one of Boltzmann - does not directly relate the H-function to whatever may be the 
entropy. It only ensures that the choice of the initial condition and some ad hoc 
nonmechanical hypotheses (a Stosszahlanzatz) gives a time-arrow and relaxation 
towards final equilibrium. 

(v) There is no difficulty with Liouville theorem of invariance of extension in 
phase space; this is a purely mechanical result. At the statistical level the extent 
of the volume of space points, compatible with the macroscopic (or mesoscopic) 
description, changes because these constraints change in time, and the character- 
istic set of microscopic points changes. The evolution of such set is a different 
thing that the set of trajectories of given points in a volume of phase space, whose 
volume is indeed conserved according to Liouville theorem. Moreover, 

(vi) It ought to be noticed that Gibbs' entropy is in fact constant in time, be- 
cause NESEF conserves the initial information, being then the said fine-grained 
entropy. But, as Fig. 1 shows, as time elapses it gets outside the informational 
subspace and therefore is no longer describing the state of the system in terms 
of the chosen set of basic variables. It needs be projected at each time on the 
informational subspace, again as shown in Fig. 1, thus introducing loss of infor- 
mation. Of course equilibrium is a particular case when Gibbs' entropy coincides 
with Clausius thermodynamic entropy, and in the NESEF formalism it remains 
always at the point of initial preparation (which is the one of equilibrium). 

(vii) Bricmont makes a similar statement to that of Meixner: Why should one 
worry so much about entropy for nonequilibrium states'? To account for the irre- 
versible behavior of the macroscopic variables is not necessary to introduce some 
kind of entropy function that evolves monotonically in time. It is not required to 
account for irreversibility, however it may be interesting or useful to do so. In the 
case we have presented of informational entropies applies this question of interest 



and usefulness: it allows for a better clarification of the meaning and interpre- 
tation of the Lagrange multipliers; to introduce the production of informational 
entropy and the derivation of useful criteria for evolution and stability; to bet- 
ter characterize the dissipative processes that develops in the media (organizing 
them in increasing orders of covariances of the informational-entropy production 
operator); to better analyze fluctuations out of equilibrium and work out studies 
on complementarity of micro/macro descriptions; and so on (see Ref. 1^ for 
details). 

Therefore, in principle and to all appearances, a true thermodynamic entropy 
is only clearly defined, via Clausius approach, in strictly equilibrium conditions. 
Out of equilibrium, quasi-entropies (in our nomenclature) may be introduced and 
be of utility, but it needs be clearly stated which is the definition, and to devise 
a particular name characterizing each: say, entropy in Classical (Onsagerian) 
Irreversible Thermodynamics, entropy in Extended Irreversible Thermodynamics, 
NESEF-entropy of Informational Statistical Thermodynamics, and so on and so 
forth 

The informational-entropy, to be related to the one in phenomenological the- 
ories, depends on a chosen set of basic variables and arises from a projected part 
of the logarithm of the said nonequilibrium statistical operator In g^{t), which 
is ln^(t, 0) (as shown in Fig. 1). This process is a coarse-grained-type procedure 
which restricts us to have as only accessible microstates in phase space those in the 
subspace spanned by the chosen basic dynamic variables. Irreversible effects are 
contained in the complementary part of the nonequilibrium statistical operator, 
namely g'^, as it is demonstrated by the proven generalized 7i-theorem presented 
in Ref. I^. 

Furthermore, the local informational-entropy production function is predom- 
inantly positive definite, but in any case these results cannot be connected with 
the formulation of the second law, until a clear cut definition of the entropy func- 
tion in nonlinear thermodynamics for systems arbitrarily away from equilibrium 
is obtained. This statement has to be further clarified since here the second law 
must necessarily be understood as some extension of Clausius formulation, with 
the latter, we recall, involving changes between two equilibrium states and 5* is 
the calorimetric entropy which is uniquely defined. We emphasize once more that 
in nonequilibrium states it is very likely that many different definitions of surro- 
gates of the entropy are feasible depending essentially on how to obtain - in some 
sense - a complete set of macrovariables that may unequivocally characterize 
the macrostate of the system under the given experimental conditions, and thus 



agreeing with Meixner conjecture jH2]- In the Informational Statistical Thermo- 
dynamics - which can be considered as given microscopic, i.e. mechano-statistical 
foundations, to macroscopic Extended Thermodynamics -, the associated infor- 
mational entropy (or quasi-entropy) has been discussed in detail in Refs. [SBl HE] , 
where are presented the derivation: of nonequilibrium equations of state; of a 
generalized ?i-theorem; of evolution and (in)stability criteria; of a generalized 
Clausius relation; of fluctuations and Maxwell-like relations; of a Boltzmann-like 
relation: S (t) = ks \nW (t); and comparison of theory and experimental results 
is made. 

As a results of this, it may be stated that NESEF seems to offer a sound formal- 
ism to give foundations to irreversible thermodynamics on a statistical-mechanical 
basis, an approach providing what has been referred-to as Informational Statistical 
Thermodynamics. Thus, for the case of systems under quite arbitrary dissipative 
conditions (no restriction to local equilibrium, linearity, etc.) a theoretical treat- 
ment of a very large scope follows for the thermodynamics, transport properties, 
and response functions of nonequilibrium systems. Paraphrasing Zwanzig [M] , we 
remark that, seemingly, NESEF possesses a remarkable compactness and has by 
far a most appealing structure, being a very effective method for dealing with non- 
linear and nonlocal in space and time transport processes in far-from-equilibrium 
many-body systems. 

4. COMMENTS AND CONCLUDING REMARKS 

Summarizing, we first notice the relevant point that in the construction of a 
statistical mechanics, the derivation of an appropriate (for the problem in hands) 
probability distribution - associated to a set of constraints imposed on the system 
- can be obtained in a compact and practical way by means of optimization 
(variational) principles in a context related to information theory. These are 
methods of maximization of the so-called informational-entropies (better called 
quantities of uncertainty of information) or minimization of distances in a space 
of probability distributions (MaxEnt and MinxEnt respectively) [22] ■ 

In the original formulation of Shannon and Jaynes use was made of Boltzmann- 
Gibbs statistical-entropy, which in MaxEnt provides the canonical-like (exponen- 
tial) distributions of classical, relativistic, and quantum statistical mechanics. In 
Ref. it is described its use for the case of many-body systems arbitrarily far 
removed from equilibrium, and the discussion of the dissipative phenomena that 
unfold in such conditions (mainly ultrafast relaxation processes; see Ref. |3U]). 



This approach has been exceedingly successful in conditions of equilibrium, and 
is a very promising one for nonequilibrium conditions. To have a reliable statistical 
theory in these situations is highly desirable since in very many situations - as for 
example are the case of electronic and optoelectronic devices, chemical reactors, 
fluid motion, and so on - the system is working in far-from-equilibrium conditions. 

However the enormous success and large application of Shannon- Jaynes method 

to 

Laplace-Maxwell-Boltzmann-Gibbs statistical foundations of physics, as it has 
been noticed, some cases look as difficult to be properly handled within the 
Boltzmann-Gibbs formulation, as a result of existing some kind of fuzziness in 
data or information, that is, the presence of a condition of insufficiency in the 
characterization of the (microscopic and/either macroscopic or mesoscopic) state 
of the system. Such, say, difficulty with the proper characterization of the system 
in the problem in hands, (which is a practical one and, we stress, not intrinsic 
to the most general and complete Boltzmann-Gibbs formalism) can be, as shown, 
patched with the introduction of peculiar parameter-dependent alternative struc- 
tural informational-entropies (see Table I). 

Particularly, to deal with systems with some kind of fractal-like structure the 
use of Boltzmann-Gibbs-Shannon- Jaynes infoentropy would require to introduce 
as information the highly correlated conditions that are in that case present. Two 
examples in condensed matter physics are "anomalous" diffusion and "anoma- 
lous" optical spectroscopy when fractality enters via the non-smooth topogra- 
phy of the boundary surfaces which have large influence on phenomena occurring 
in constrained geometries (nanometer scales in the active region of the sample). 
In the conventional approach, the spatial correlations that the granular boundary 
conditions introduce need be given as information (to satisfy the criterion of suf- 
ficiency, since they are quite relevant for determining the behavior of the system 
in the nanometric scales involved), but to handle them is generally a nonfeasible 
task. For example, in the second case above mentioned one has no easy access 
to the determination of the detailed topography of the surfaces which limit the 
active region of the sample (the nanometric quantum wells in semiconductor het- 
erostructures) , what can be done in the first case using atomic-force microscopy 
and the determination of the fractal dimension involved is possible. Hence the 
most general and complete Boltzmann-Gibbs formalism in Shannon- Jaynes ap- 
proach becomes hampered out and is difficult to handle, and then, as shown, 
use of other types of informational-entropies (better called generating functionals 
for deriving probability distributions) may help to circumvent such inconveniency 



by introducing alternative algorithms (dependent on the so-called informational- 
entropic indexes), that is, the derivation of heterotypical probability distributions 
on the basis of the constrained maximization of unconventional informational- 
statistical entropies (quantity of uncertainty of information), to be accompanied, 
as noticed in the main text, with the use of the so-called escort probabihties. 

In brief, we recall that. Unconventional Statistical Mechanics consists of two 
steps: 1. The choice of a deemed appropriate structural informational- entropy 
for generating the heterotypical statistical operator, and 2. The use of a escort 
probability in terms of the heterotypical distribution of item 1. 

We reiterate that the escort probability introduces corrections to the insuffi- 
cient description by including correlations and higher-order variances of the ob- 
servables involved. On the other hand, the heterotypical distribution introduces 
corrections to the insufficient description (or incomplete probabilities in Renyi's 
nomenclature) by modifying the statistical weight of the dynamical states of the 
conventional approach involved in the situation under consideration. 

Moreover, we have considered a particular case, namely the statistics as derived 
from the use of Renyi informational entropy We centered the attention on the 
derivation of an Unconventional Statistical Mechanics appropriate for dealing with 
far-removed-from-equilibrium systems. 

In conclusion, we may say that USM appears as a valuable approach, in which 
the introduction of informational-entropic-indexes-dependent informational-entro- 
pies leads to a particularly convenient and sophisticated tool for fitting theory to 
experimental data for certain classes of physical systems, for which the criterion 
of sufficiency in its characterization cannot be properly satisfied. Among them we 
can pinpoint fractal-like structured nanometric-scale systems, which, otherwise, 
would be difficult to deal with within the framework of the conventional Statistical 
Mechanics. While in the latter case one would need to have a detailed description 
of the spatial characteristics of the structure of the system, the other needs to pay 
the price of having an open adjustable index to be fixed by best fitting with ex- 
perimental results. It is relevant to notice the fact that the infoentropic index(es) 
is(are) dependent on the dynamics involved, the system's geometry and dimen- 
sions, boundary conditions, its macroscopic-thermodynamic state (in equilibrium, 
or out of it when becomes a function of time), and the experimental protocol. 

Finally, we call the attention to the fact that we have presented several al- 
ternatives of infoentropies (see Table I), for which, as stated in the main text, 
the uniform probability distribution is taken as the reference one, and such gen- 
erating functionals provide a corresponding family of heterotypical probability 



distributions. However, other choices of the reference probabihty can be made 
and then wc have at our disposal very-many possibihties: It is tempting to look 
for the construction of a theory using for the probability of reference, instead of 
the uniform distribution. Shannon- Jaynes informational-entropy in its incomplete 
formalism, that is, when suffering from the deficiency that the researcher cannot 
satisfy Fisher's criterion of sufficiency. 
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;ure 1: An outline of the description of the non-equihbrium-dissipative macro- 
scopic state of the system. The projection - depending on the instantaneous 
macrostate of the system - introduces the coarse-graining procedure consist- 
ing into the projection onto the subspace of the "relevant" variables associ- 
ated to the informational constraints in NESEF. 



